Technical Documentation
TRIM3 is a client/server system. It can also be viewed as a three-tiered system, with the client as one tier, and the server as two -- one the application software and the other the database. Since TRIM3 has a web-based interface, the only software required on the client is a web-browser. The bulk of TRIM3's work is done by the servers. The TRIM3 system uses two servers, one for maintaining the version of TRIM3 which is available to the public (the public server), and one for maintaining the ASPE version (the ASPE-TRIM server). A diagram of the major components of TRIM3, the type of data available, and how they are accessed by various types of users may be seen by clicking here [Word format]. For a list of hardware and software requirements, click here [Word format].
This documentation describes how the three major components of the TRIM3 system are implemented:
This documentation also describes certain policies and procedures that are observed by the current TRIM programming staff, which have proven valuable to the everyday maintenance of the TRIM system:
The Interface
TRIM3's web-based interface uses the Microsoft Internet Information Server (IIS). The many web pages that provide TRIM3 with its extensive functionality have been developed with PHP. The site "trim.urban.org" sends the user to the introductory page "T3Welcome.php". From this point, the user can link to the various other pages of the interface.
Some of the links are to simple pages containing documentation and/or general information:
- What's New (T3New.php) shows recent changes to the TRIM3 system.
- Introduction (T3Welcome.php) provides an overview of microsimulation and TRIM3.
- Documentation (T3Technical.php) provides links to model documentation.
- Staff (T3Staff.php) provides a list of core staff members.
- Contact us (T3Contact.php) provides email addresses that users can submit questions to.
- Management (T3Management.php) provides links to the configuration management and version control tools in the Calendar subdirectory (only accessible to programming staff).
- Register (Registration/index.php) sends the user to a on-line registration form, which must be completed before a username may be obtained.
The most important link is to the TRIM3 Navigator (T3Browse.php). The Navigator is the part of the interface through which most of the user's interaction with TRIM will occur, and it is the most complicated part of the interface. It provides links to six major tools:
- Data Dictionary (Dictionary/index.php) Browse term definitions.
- Program Rules (PublicRules/index.php) View the baseline values of any program rule classified as public-use data.
- Microdata (Microdata/index.php) View, extract and/or summarize microdata.
- Simulation Runs (Runs/index.php) Set-up and run simulations.
- Summary Tables (SummaryTables/index.php) View summary tables from TRIM3 simulations.
- CBasic [Word format diagram].
- CPage [Word format diagram]
- CSelector [Word format diagram]
The remaining Navigator tool (Microdata) uses classes from an in-house system called OOQM (Object-Oriented Question Maker).
The Database
All of TRIM3's data and program rules are stored in a relational database using MySQL sofware. All database objects are stored under one of three types of databases: Central TRIM Dictionary (CTD), Input, or Results.The CTD database contains the following:
- The dictionary of all TRIM3 data elements and their possible values. Not only is this useful for documentation, but the information it contains may be used to create any of the program rules, input, or microdata results tables.
- Descriptions of all simulation runs defined by users. This information is used by the TRIM3 engine, and links program rules with microdata results.
- Program rules (instructions) for all simulations.
- Data used in converting the raw CPS into a TRIM3 input database.
Input databases store microdata imported into TRIM3 from statistical surveys, such as the CPS. There is an individual database for each year's data, and well as for each version of input data for that year.
The Results database stores the two types of data which are the output of TRIM3 simulations: summary tables and microdata results. The summary tables give an aggregate summary of the simulation, while the microdata give detailed variables at levels which correspond to the Input database used by the simulation.
The flow of data between each of the three types of databases and the other components of the TRIM3 system may be seen by clicking here [Word format].
Detailed descriptions of the tables contained in each of the three types of databases may be seen by clicking below:
- CTD [Word format]
- Input [Word format]
- Results [Word format]
The Engine
The software component of TRIM3 that processes the input data and program rules, applies the rules to the data, and calculates the simulation results, is called the simulation engine. It is implemented using Microsoft's Visual C++, and has two major components: The frame supplies to each simulation module the services it needs at run-time (e.g. the frame handles the reading of input microdata and program rules, and the writing of microresults and summary table data), and supplies objects that provide the basic functionality needed by the simulation modules. The simulation modules, in turn, are created by deriving module-specific objects from these frame-supplied base classes. These module-specific objects apply the appropriate program rules to the input data in order to simulate the corresponding welfare, tax, and/or health programs. A diagram (Word format) showing how the frame and simulation modules interact may be seen by clicking here. The frame is also responsible for converting raw survey files into TRIM3 input files.The Frame
The frame is implemented using object-oriented programming. The source code is divided into three groups:- The TRIMexe group contains the code needed to run a simple dialog process on the server. This process listens for requests from clients, and then gives the commands necessary to begin fulfilling the request.
- The Convert group contains the source code for converting a survey into a TRIM3 input file (PDF diagram) .
- The TRIMdll group contains the bulk of the frame's source code. This includes code for reading and writing of microdata, reading program rules, creating a "make" file, and suppling the basic functionality needed by the simulation modules. UML diagrams showing the relationships between some of the classes defined in the TRIMdll group may be viewed below:
Simulation Modules
As mentioned above, each simulation module is implemented by deriving module-specific objects from the base classes provided by the frame. A simulation module must provide the frame with five classes derived from the following base classes (and overwriting the specified virtual function):- CSim (overwrite Simulate)
- CUnit (overwrite SelectUnit)
- CResultSet (overwrite CopyResultRecord)
- CResultMonthlySet (overwrite CopyResultMonthlyRecord)
- CResultIterator (Specify all the output variables)
- CEligible
- CAssets
- CIncome
- CBenefit
- CParticipate
Backup Policy
The TRIM3 backup strategy is based on preventing the loss of program rules. It is rarely necessary to store any backup copies of results microdata since they may be re-created simply by running the simulation again with the stored values of its program rules. Backups of the full program rules database are done each day for each TRIM3 server. Daily backups are available on disk for four weeks and are archived to tape afterwards. The only simulation results ever backed up are annual baselines, which are static and are used as input to a number of alternative simulations. Backup copies are made as soon as the baselines for a given year are finalized.The TRIM3 database backup procedures for a single server are as follows:
| Action | Frequency | Description |
| Full backup of program rules data | Daily | Backup of all program rules necessary to run all simulations currently defined on the server. Available for four weeks on disk. Archived to tape afterwards. |
| Backup of baseline results microdata | On finalization of baselines for a given year | Backed up separately to allow fast recovery since many simulations use this data as input. |
| Backup of input microdata | On conversion of CPS data for a given year | Backed up when the CPS conversion results have been verified. |
| Full cold backup of server | On structural changes in the database | Performed whenever the configuration of the database changes. |
A backup server is maintained with the latest version of the simulation engine, input microdata and baseline microdata results. The backup server does not maintain an up-to-date version of the program rules due to the frequency of changes made to them. Rather than immediately updating program rules on the backup server each time a change is made on the production server, the program rules on the backup server are only updated if the production server experiences an unrecoverable crash. In that case, the program rules will be restored to the backup server from the previous day's backup of the production server (this process may be completed in a matter of minutes). While this means users will have to re-enter any changes made to the rules that day, it avoids the increased maintenance costs of having the backup server "mirror" the production server. Furthermore, since the beginning of TRIM3's use as a production system, we have experienced no significant down time of the production server.
Security Policy
Much of the data available through the TRIM3 system is considered public-use, and hence there are no restrictions on access to it. Users of this public-use TRIM data only need to register before actually obtaining the data. When registering, users are asked for some basic information about who they are, how they may be contacted, and what their expected uses of the data will be. At that point, they are assigned a username and password which will give them access to the public data but which does not allow the user to run his/her own simulations.To access TRIM3 data that are not classified as public-use, and/or to run simulations, a user must have their username's access upgraded. This upgrade can only be done by UI staff. All upgraded users are placed into user-groups, which determines an individual user's access to runs and data owned by users in another group. Unless specific exceptions are requested, users in one user group do not have access to runs or data owned by users in another group. The HHS project officer must approve requests for access to non-public data for any persons other than TRIM3 project staff.
Currently, TRIM3's security is implemented using two servers. Users with access to non-public data are directed to a different server than public-data users. The data available on the public-use server is a subset of the data available on the non-public server. As ASPE approves data for public use, it is copied from the non-public server to the public server.
Version Control and Testing
All versions of the engine source code ever released to the server, along with descriptions, are saved in the TRIM CTD database and assigned a major and minor version number. The corresponding compiled DLL are saved in the database under that version number as well. This allows the simulation engine to retrieve the executable DLL for a given version on request. The TRIM3 policy is (to the extent possible) to maintain backward compatibility among different versions. This guarantees that results of an older version of the model may be reproduced in the future, while still enabling users to re-run older runs using updated versions of the simulation code.The web interface, as well as the structure of the database (as opposed to the data in the database) are subject to significantly fewer changes than the simulation code. Furthermore, once an upgrade is made to one of these components, there is generally no need to go back to earlier versions. Consequently, while a log of changes to these components is kept, previous versions are not retained. However, changes made by users to the values of program rules are kept in a log. This log may be viewed by users, thus allowing them to see the history of changes made to the program rule values of a particular module's run.
When a change is made to one of the TRIM3 simulation modules, testing of the module is performed by the developer on the developer's PC, which is configured to run as a test server. After the module has been tested, the changed version is moved to the production server, but is not made available to general users. During this phase, integeration testing is performed by the developer, and acceptance testing is carried out by selected policy analysts. After both of these tests are completed, the change is announced to the user community and made available for general use.
TRIM3 ©2012